Probability-Enhanced Predictions in the Anticipatory Classifier System

In contrast to common Learning Classiier Systems (LCSs), classiiers in the Anticipatory Classiier System (ACS) have a condition-action-anticipation-payoo structure (Stolzmann, 1998). The learning is based on the accuracy of predicted environmental eeects (i.e. anticipations) rather than on the payoo predictions, as in traditional LCSs, or the accuracy of payoo predictions, as in XCS (Wilson, 1995). Anticipation based learning enables the ACS to learn latently (i.e. learning without getting any reward) a complete internal model of the environment rather than a condition-action-payoo model. Additionally to the evolution of an internal environmental model, the ACS applies reinforcement learning resulting in the formation of a behavioral policy. The evolving environmental model enables the ACS to use more sophisticated processes (i.e. cognitive processes) to solve more diicult tasks, speed up learning, and reene its behavioral policy. Stolzmann, Butz, Hoomann, and Goldberg (2000) investigate the present cognitive capabilities. Figure 1 visualizes the learning in the ACS. Fig. 1. A behavioral act in the ACS with all the involved learning procedures Similar to all LCSs, the ACS evolves a population of classiiers. The main learning mechanism is the Anticipatory Learning Process (ALP) which is derived from a cognitive learning theory. It generates oospring classiiers by specializing inaccurate, more general ones. The recently introduced genetic generalization (Butz, Gold-berg, & Stolzmann, 2000) evolves accurate, maximally general classiiers out of the partly over-specialized classiiers created by the ALP. Additionally , the reinforcement learning component, which is a combination of the Bucket-Brigade and Q-learning, evolves a behavioral policy in the ACS. In order to be able to learn an internal environmental model, the ACS relies on a causality in successive perceptions of an environment. Any non-determinism challenges the learning mechanism since until now the ACS considered all changes in the environment to be caused by itself. Herein, we consequently investigate the possibilities in the ACS to enhance the predictions. We enhance the eeect part of a classiier to probability-enhanced eeects (PEEs)